Test Topics
Step 3. Create the Test Specifications
Introduction
After the overall content of the test has been established through a job analysis, the next step in test development is to create the detailed test specifications. Test specifications usually include a test description component and a test blueprint component. The test description specifies aspects of the planned test such as the test purpose, the target examinee population, the overall test length, and more. The test blueprint, sometimes also called the table of specifications, provides a listing of the major content areas and cognitive levels intended to be included on each test form. It also includes the number of items each test form should include within each of these content and cognitive areas.
Components of the Test Specifications
Test Description
The test description component of an exam program's test specifications is a written document that provides essential background information about the planned exam program. This information is then used to focus and guide the remaining steps in the test development process. At a minimum, the test description may simply indicate who will be tested and what the purpose of the exam program is. More often, the test description will usually also include elements such as the overall test length, the test administration time limit, and the item types that are expected to be used (e.g., multiple choice, essay). In some cases the test description may also specify a test administration mode (e.g., paper-and-pencil, performance-based, computer-based). And, if the test will include any items or tasks that will need to be scored by human raters, the test description may also include plans for the scoring procedures and scoring rubrics.
Test Blueprint
The content areas listed in the test blueprint, or table of specifications, are frequently drawn directly from the results of a job analysis. These content areas comprise the knowledge, skills, and abilities that have been determined to be the essential elements of competency for the job or occupation being assessed. In addition to the listing of content areas, the test blueprint specifies the number or proportion of items that are planned to be included on each test form for each content area. These proportions reflect the relative importance of each content area to competency in the occupation.
Most test blueprints also indicate the levels of cognitive processing that the examinees will be expected to use in responding to specific items (e.g., Knowledge, Application). It is critical that your test blueprint and test items include a substantial proportion of items targeted above the Knowledge-level of cognition. A typical test blueprint is presented in a two-way matrix with the content areas listed in the table rows and the cognitive processes in the table columns. The total number of items specified for each column indicates the proportional plan for each cognitive level on the overall test, just as the total number of items for each row indicates the proportional emphasis of each content area.
The test blueprint is used to guide and target item writing as well as for test form assembly. Use of a test blueprint improves consistency across test forms as well as helping ensure that the goals and plans for the test are met in each operational test. An example of a test blueprint is provided next.
Example of a Test Blueprint
In the (artificial) test blueprint for a Real Estate licensure exam given below the overall test length is specified as 80 items. This relatively small test blueprint includes four major content areas for the exam (e.g., Real Estate Law). Three levels of cognitive processing are specified. These are Knowledge, Comprehension, and Application.
Each test form written to this table of specifications will include 40% of the total test (or 32 items) in the content area of Real Estate Law. In addressing cognitive levels, 35% of the overall test (or 28 items) will be included at the Knowledge-level. The interior cells of the table indicate the number of items that are intended to be on the test from each content and cognitive area combination. For example, the test form will include 16 items at the Knowledge-level in the content area of Real Estate Law.
Content |
Knowledge |
Comprehension |
Application |
Total |
Percentage |
Real Estate Law |
16 |
8 |
8 |
32 |
40% |
Real Estate Practices |
4 |
12 |
|
16 |
20% |
Financing/ |
8 |
8 |
8 |
24 |
30% |
Real Estate Math |
|
|
8 |
8 |
10% |
Total |
28 |
28 |
24 |
80 |
|
Percentage |
35% |
35% |
30% |
|
100% |
Summary
The test specifications for an exam program provide essential planning materials for the test development process. Thorough, thoughtful test specifications can guide the remainder of the test development process, especially item writing efforts and test assembly. An initial test form can be developed according to these specifications to appropriately reflect the content and cognitive emphases intended. The specifications can also be used to guide the development of later, additional test forms. Careful linking between the job analysis, test specifications, and test items will go a long way to providing strong content validity and legal defensibility for the exam program.